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Abstract 

As freely adoptable digital resources, OpenCourseWare (OCW) have become a prominent form of Open 
Educational Resources (OER). More than 275 institutions in the worldwide OCW consortium have 
committed to creating free access open course materials. Despite the resources and efforts to create OCW 
worldwide, little understanding of its use exists. This paper reports OCW project development at National 
Taiwan University (NTU) and investigates its use with Google Analytics. Reports include strategic plans to 
overcome challenges to OCW creation and implementation, the project’s growth and maturation, overall 
use of OCW, and possible future directions. As a result of its 5-year development and of lessons learned, the 
NTU-OCW experience features: (1) integrating resources on campus and established operating procedures, 
(2) setting course selection criteria featuring the strength of NTU and Taiwan, (3) providing coherent 
program support to enhance faculty participation, and (4) adhering strictly to the Creative Commons 
license. Data from Google Analytics was reviewed for better understanding of the use, characteristics, 
course preferences, and behaviors of NTU-OCW users. Results show visitors were primarily lifelong 
learners (65%) in informal learning settings. Statistics indicate an overall successful use of NTU-OCW for 
Chinese speaking users, especially in urban areas where information and communication technology is 
more developed. Potential impacts and future improvements are discussed, including how to promote 
usage of OCW courses for on and off campus users, adding rating features and indexing for customizing 
search, and integrating OCW into the learning management system (LMS) as part of OER. 

Keywords: open education, open educational resources (OER), online learning, informal learning, OCW 
concept, higher education 
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Introduction 

In the global movement of Open Education Resources (OER), OpenCourseWare (OCW) is an important 
force. An OCW is defined as “a free and open digital publication of high quality educational materials, 
organized as a course” (Carson, 2009, p. 27). These resources are “free and openly licensed, accessible to 
anyone, anytime via the internet” (Open Education Consortium, n.d.-a, para. 1). Since the Massachusetts 
Institute of Technology (MIT) initiated its OCW project in 2001 and inspired others to follow, 275 
institutions around the world have joined the Open Education Consortium (OEC) to share the common 
values of openness, equity, collaboration, and multiculturalism (Open Education Consortium, n.d.-b). 
However, like every educational movement, every OCW project differs from one to another due to unique 
educational settings. To better promote access to education, to improve the quality of education, and to 
empower both well-served and underserved populations through the power of open learning, regional 
consortiums or organizations formed over the years. OEC sustaining members include among many others 
the African Virtual University (AVU) for Pan African, Universia (a Latin American, Spanish and Portuguese 
network), Japan OCW consortium, Korean OCW consortium, and Taiwan Open Course Consortium in Asia. 

As one of Taiwan’s leading universities, National Taiwan University (NTU) is known for its high-quality 
teaching, learning resources, and richness of course content. NTU also shares societal responsibilities in 
providing quality education, conducting higher level research, and promoting open education models for 
others to follow. Inspired by and joining the world-wide open education mission, NTU adopted the OCW 
concept to promote open education and to share knowledge by making educational resources available 
through the Internet. In November 2008, NTU joined the Taiwan Open Course Consortium (TOCC), and 
began research and discussions on developing a platform for open educational resources. As a result, NTU 
began the OCW project in 2010. In July 2011, NTU became a member of the OCW Consortium (now Open 
Education Consortium) and was committed to developing an OCW website with high quality course 
materials. Through OCW, NTU can share its educational resources with the public, and enhance the 
participation of teachers and students in creative teaching and learning. With its quality-first principle, 
consistent effort, and collaboration from all involved individuals, the NTU OCW site reached the 12-million- 
visitor mark in 2016. In addition, its website was named the “Outstanding Site” by OEC in 2014. During the 
following year, an NTU OCW faculty member received the “Educator Award” of the Open Education Awards 
for Excellence. 

However, realizing the great potential of OCW concept requires consideration of many issues. Some are 
global, and some are unique to individual institutions or to specific geographic regions. While most previous 
studies about OCW focus on initiatives in a western context, there is a need to explore and share successful 
OCW experiences in Asia, specifically in Taiwan. From the perspective of NTU-OCW, several questions 
should be answered: Would the OCW concept fit NTU’s core values and characters? What could NTU best 
offer, considering its unique position in Taiwan and the world? To what extent has it used? By whom and 
how? How could the greatest benefit with limited resources be provided locally and globally? As a case study, 
this paper presents the situations NTU faced while adopting the OCW concept in creating open educational 
resources. The paper shares NTU’s experiences with implementation and promotion of online learning 
materials, specifically OCW courses and discusses OCW use and provides suggestions for producing high 
quality online learning resources. 
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This paper will introduce key concepts from the literature related to open education, OER, and OCW. Next, 
it will detail the development and implementation of the NTU OCW experience. Google Analytics will 
provide insights into the uses, characteristics, course preferences, and behaviors of NTU-OCW users. 
Finally, the implementation of findings and future direction will be discussed. 


Literature Review 

Discussions about OER have accelerated during the past decade and related initiatives are being launched 
around the world (Guo, Zhang, Bonk, & Li, 2015). Previous research is discussed from four perspectives: (1) 
OER, (2) OCW, (3) OER, and OCW in Taiwan. 

Open Education Resources 

Rapidly developing Internet technology continues to bring new opportunities to education. One promising 
and exciting example is OER. While the term "open educational resources" was first adopted by UNESCO 
in 2002, the idea of open education emerged in the late 20 th century from developments in distance learning. 
The culture of open knowledge and open sharing allows easy access and easy reuse of teaching and learning 
materials to a wide range of educators and learners. OER typically refers to digital resources, including 
articles, e-text, video, audio, modules, full or partial courses, and any learning materials (Wiley, Bliss, & 
McEwen, 2014). However, based on UNESCO’s definition, OER are “resources for teaching, learning and 
investigation which are in a public domain site or have been published under an intellectual property license, 
allowing other people its free use or for different purposes than those referred by its author” (Pelaez, 
Pullaguari, & Caro, 2011, p. 1075). It indicates that the idea of OER includes content material, the 
technological tools to support or deliver it, and related issues of legal usages. OER can be created by anyone 
or any organization willing to participate in common open knowledge. To be open, OER materials usually 
are released under intellectual property regulations such as Creative Commons that support open use of 
content (Educause, 2010). OpenCourseWare, Connexions, and the Open Learning Initiatives are some of 
the well-known OER projects. 

Given the philosophy of OER to be open and make education more accessible, OER is one way to address 
the rising costs of education (Educause, 2010; Wiley, Green, & Soares, 2012). Many recent studies have 
discussed the potential of OER materials and resources replacing traditional paper-based textbooks. 
Hunsicker-Walburn, Guyot, Meier, and Beavers (2016), for instance, investigated community students’ 
preferences of OER textbooks. More than half the students (64%) indicated textbook costs would impact 
their perception of attending higher learning. In addition, in their study, approximately 93% students felt 
satisfied with OER textbooks. Similar results were found in Delimont, Turtle, Bennett, Adhikari, and 
Lindshield (2016) which surveyed and interviewed students and faculty in 13 university courses that used 
OER. They found that all students and nearly all faculty had positive preferences toward using OER. Hilton 
(2016) reviews 16 additional studies pertaining to OER efficacy and perceptions and find that generally, 
OER can be successfully substituted for traditional learning resources. In another study, Watson, Clouser, 
and Domizi (2014) shared redesign strategies, financial benefits, and pedagogical opportunities from two 
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courses that have replaced textbooks with OER. They concluded that collective advantages provide a 
compelling argument for broad adoption of OER in higher education. 

Despite such positive research results related to the use of OER in instruction, several issues emerged with 
the increasing volume of OER creation and its usage. Such issues include the need for effective search 
methods to find a proper or suitable OER, quality control of OER creation, and evaluation mechanisms for 
OER. Groom (2013) indicated that decreasing the technical and metadata requirements of finding and 
sharing OER can enhance teachers’ levels of participation in OER. Given that anyone can create OER, the 
quality of the OER is not always consistent. In most cases, OER are developed by individuals who may not 
have proper training in instructional design or who participate in fairly casual and unstructured online 
collaboration (Daniel, Kanwar, & Uvalic-Trumbic, 2009). In addition, Dhanarajan and Abeywardena (2013) 
find that faculty member’s knowledge and skills of evaluating OER quality can impact their adoption and 
use. 

OpenCourseWare 

Although OER and OCW are sometimes used interchangeably in studies, the two differ. OCW usually 
includes syllabus, handouts, course slides, lecture videos, and other teaching and learning materials that 
are “organized as courses.” OER resources can range from “any digitized form” (Terrell & Caudill, 2012, 
p.39.). Therefore, OCW is part of a larger subset of OER. In the global OER movement, the MIT OCW has 
continued to play an essential role since its inception in April 2001 (MIT, 2001). 

Today, MIT OCW stands as one of the longest-running OER projects in the world (Educause, 2010). While 
several leading universities considered offering distance learning programs to a world community for profit 
by relying on their famous brands and resources, the MIT OCW project created a non-profit model for open 
knowledge and open sharing (Carson, 2009; Shumar, 1997). In June 2001, MIT announced the launch of 
two projects to make approximately 2,000 lecture materials available online with an open license. That 
means learners around the world can access high-quality course resources that are “free and openly licensed, 
accessible to anyone, anytime via the internet” (Open Education Consortium, n.d.-a). To date, MIT OCW 
website has 200 million visits from users around the world. Among these users, 43% identified themselves 
as self-learners, 42% as educators, and 9% as students (MIT OpenCourseWare, 2016, para. 1). The dual 
missions of MIT OCW clearly expressed the concept of OCW, “to provide free access to virtually all MIT 
course materials for educators, students and individual learners around the world and to extend the reach 
and impact of MIT OpenCourseWare and the OpenCourseWare concept” (Carson, 2009, p. 26). OCW 
shares educational resources and course materials to anyone who wants and needs it, and the OCW 
movement promotes the idea of knowledge sharing itself. 

The growth of OCW inspired other institutions around the globe to join the OER movement. The Open 
Education Consortium (former OCW consortium) was established in 2005 and was intended to reintegrate 
educational resources, promote higher education and contribute to global knowledge sharing. Following 
that model, regional consortiums began to emerge including Universia to offer the Latin American, Spanish, 
and Portuguese community, African Virtual University (AVU), the leading Pan-African Open Distance and 
eLearning Network to facilitate the use of open, distance, and eLearning in African institutions, as well as 
Japan OCW Consortium and Taiwan Open Course Consortium (TOCC) in Asia. 
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While the idea of OCW was broadly accepted around the world, its actual usage and impact on learning 
creates some doubts. A common criticism of OCW is its one-way delivery of information. Although content 
materials are organized into courses, OCW lacks interactions among teachers, students and peers common 
in classrooms. Cueva, Rodriguez, and Marban’s (2016) study used quantitative and qualitative methods to 
investigate the level of acceptance and expectations of integrating social technologies in OCW. They 
concluded that with implementation of collaborative learning design and supporting technologies, the 
effectiveness of social and collaborative learning in OCW can be enhanced. Massive Open Online Course 
(MOOC) that provide certain kinds of interactions between students with teachers and among peers, or 
Small Private Online Course (SPOC) that combine OCW and classroom are additional ways to use OCW 
materials interactively. 

Many important issues for OER and OCW include design, development, accessibility, suitability, 
distribution, selection and quality control. One major issue of concern for both OER and OCW is the quality 
of materials. Given the increasing creation, availability, delivery, and use of OER materials and OCW 
courses under the concept of educational openness, two emerging issues are how to build a quality model 
and how to designate criteria for online learning content. Many previous studies focus on the effectiveness 
and quality aspects of MOOCs. Walker and Loch’s (2014) survey with academics who participate in MOOC 
found that the delivery quality of MOOC does impact academics’ perception to MOOC. Video quality, 
interaction, and assessment are some aspects mentioned in their survey results. Yousef, Chatti, Schroeder, 
and Wosnitza (2014) examine the criteria of effective MOOC and found that among all the criteria, learning 
analytics and assessment were identified as the most important ones. Among all the quality assurance 
frameworks for online learning that are proposed by various researchers, one of the most popular and 
widely used is the Quality Matters Program (QM). QM suggests eight quality dimensions for online 
learning: course overview and introduction, learning objectives, assessment and measurement, 
instructional materials, learner interaction and engagement, course technology, learner support, and 
accessibility. However, because these factors are for general online learning, they might not be suitable for 
OCW due to its unique features. For example, OCWs are delivered in a synchronized environment and 
usually provide only one-way interaction of information. Evaluation or assessment elements are not 
included. 

An OCW study indicated that economic consideration and the ordering of intellectual property rights are 
two challenges to its adoption (Pelaez et al., 2011). Tovar, Piedra, Chicaiza, Lopez, and Martinez-Bonastre 
(2012) proposed eight key elements for generating quality OCW content: development and distribution 
models, academic scope, content development and user interfaces, evaluation of education-oriented 
support, human-computer interaction tools, evaluation of technical requirements and evaluation of 
repository use, and accessibility. Although the components of these quality models and criteria selection 
might differ within the variety of open educational formats such as OER, OCW, and recently popularized 
Massive Open Online Courses (MOOCs), all emphasize the important influences of high quality educational 
materials to learning. 

In their paper about proposing a quality model for OCW materials, Pelaez et al. (2011) identified three 
essential areas that OCW materials must address: educational, technical, and legal. Educational aspects 
refer to “teaching materials structured in adequate way with a common appearance. “ Technical aspects 


104 



Evaluating NTU’s OpenCourseWare Project with Google Analytics: User Characteristics, Course Preferences, and Usage Patterns 

Sheu and Shih 


include “an Internet site with worldwide access and with the proper quality.” Legal aspects consider “the 
open licenses application and the use of materials in a non-commercial context.” (p.1076) For effective OCW 
adoption, economic consideration and the ordering of intellectual property rights are the two main 
challenges. 

The increasing numbers of OCWs do not assure their high quality or educational utility (Porcello & Hsi, 
2013; Rodriguez, Dodero, & Alonso, 2011). Moreover, low-quality OER/OCW can hamper instructional 
practices and waste teachers’ limited time (Abramovich, Schunn, & Correnti, 2013; Wetzler et al., 2013). As 
a result, developing high quality OERhas become a critical factor in the success of online education. 

OER/OCW in Taiwan 

TOCC, established in 2008, aims to expand open education to all fields of studies and to break down 
university walls and boundaries to knowledge exchange. As a sustaining member of OEC, TOCC promotes 
OCW under the principles of openness, sharing, and collaboration, and provides diverse and rich learning 
resources for global Chinese communities. By the end of 2016, TOCC consisted of 28 members including 
27 higher-education institutes and one senior high school. One of the features of TOCC courses is that more 
than 75% are full-courses, which include lecture videos and required course materials (Wan & Lee, 2012). 

In response to the OCW wave or OCW inspiration to open education, related issues were discussed from 
various perspectives in Taiwan. Wu (2014) examined 11 empirical studies about OCW in Taiwan that were 
published in Chinese. Primary topics included advantages and disadvantages of adopting OCW, OCW 
applications in various subject areas, such as calculus (Chen & Pai, 2012), computer skills (Tsai, Shen, & 
Huang, 2012), and teacher education (Wu, 2014). Topics on successful courses, designed and development 
of OCW websites, intellectual property concerns, and the perceptions of faculty and students were discussed 
(Wu, 2014). 

Chou, Hung, Wei, and Wang (2011) probed faculty members’ readiness for OCW in Taiwan using an 8- 
factor scale, including acceptance of OCW, personal characteristics, self-perceptions of ICT competency, 
teaching method and style, effect on teaching and research, administrative support and incentive, 
intellectual property issues, and interaction with users. In addition, Wang, Chen, and Hu (2013) 
interviewed faculty members in nine Taiwanese public universities and found that faculty's beliefs of OCW 
and OER play an essential role in their participation in OCW. They also found that administrative support 
contributes to a faculty member’s willingness to use OCW materials or participate in an OCW project. 

While most of the leading universities in Taiwan devoted effort and time in OCW production, related studies 
are few, especially articles published in English. Among those few studies to date (Chen & Pai, 2012; Chou 
et al., 2011; Sun & Wu, 2016; Wan & Lee, 2012; Wang et al., 2013; Wu, 2014; Young & Hung, 2014), the 
researchers have focused on the initiatives of OCW projects and the adoptability/acceptance of OCW in 
higher education. In addition, there was a shift from OCW to MOOC in literature regarding open education 
and online learning. In 2013, Taiwan’s Ministry of Education launched a nationwide MOOC project in 15 
universities (Chen, 2013). NTU also hosted the first-ever Chinese MOOC on Coursera. Since then, the 
primary focus of OER in Taiwan has shifted from OCW to MOOC. As a result, many universities concentrate 
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on making MOOC and discontinued production of OCW. The focus of research interests also shifted to 
MOOC related topics. 

In recent years, the few remaining OCW studies are usually caused by emerging technologies, such as the 
integration of mobile devices in OCW. Young and Hung’s (2014) study discussed implementation of a 
mobile-based OCW platform and its usages. They concluded that to expand OER in Chinese societies, issues 
about intellectual property rights need further attention. They also argued that cloud-based platforms are 
one of the more important and interesting approaches for future OCW initiatives and projects. Another area 
of recent interest and attention is the effect of flipped classroom approaches to OCW (Sun & Wu, 2016; Sun, 
Wu, & Lee, 2017). Sun and Wu’s (2016) study with 181 freshmen in a course on physics found that the 
integration of OCW and flipped classrooms can help students achieve higher learning performance than the 
integration of OCW and a conventional classroom. In spite of the importance of these studies, few have 
focused on the use of OCW project and the evaluation of design and solutions in responding to local or 
global concerns or needs. 

Currently, NTU is one the few universities that continuously produce both OCW and MOOCs. Although 
there are studies discussing OCW issues in Taiwan, most were conducted before 2013 and the majority of 
them are published in Chinese. To understand the current state of OCW in the post-MOOC era in Taiwan, 
it is necessary to take a look at NTU OCW for updated information. It can provide a good reference to the 
development of OCW in non-western context. 

Purpose of the Study 

In spite of the importance of the previous research, none has focused on the use of OCW and an evaluation 
that could inform design and solutions in response to concerns and needs globally or locally. In addition, 
although the popularity of NTU OCW is continuously increasing and the quality of materials are recognized 
by peer institutions and organizations in the OER community, its current usage and factors leading to the 
achievement are rarely discussed. Therefore, the purpose of this paper is twofold. One is to contribute to 
the gap of literature about the use of OCW and the other is to examine the possible reasons contributing to 
the success of NTU case. Together, this paper offers findings that inform researchers and educators about 
the potential next steps of OCW projects and initiatives at NTU and hopefully to address a more global 
audience. 

Significance of the Study 

While many efforts have made to create OCW, there is little understanding about how OCW was used. The 
present paper provides information about the use of OCW. It fills the gaps in literature about how OCW was 
used. This paper also provides various suggestions for future directions, based on the findings from 
qualitative insights and quantitative google analytic data. We believe “next steps’ are critical for the OCW 
community after achieving its initial goals, to develop and implement OCW since its MIT inspiration. As 
such, the present paper can be a reference or a model for others on “what next” or facing similar challenges, 
including suitability, roles in open education, and design for better impact on teaching and learning. It is 
especially important for the OER community in Taiwan. 
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Development and Implementation of NTU OCW 

This section details the development and implementation of the NTU OCW experience, including: (1) the 
context of NTU-OCW, (2) the formation of NTU-OCW working group in responding to the call for action 
and challenges, (3) the features of NTU-OCW experience that resulted from the process, and finally (4) the 
NTU-OCW production model that leads to OCW success. 

Context of NTU-OCW 

National Taiwan University, one of the most prestigious universities in Taiwan, consists of 11 colleges, 54 
departments, 103 graduate institutes, and 4 research centers. The university has more than 30,000 students 
in 2015 (16,533 undergraduate students and 15,225 graduate students.) The main campus of the university 
is located in the center of Taipei, a modern urban city with an estimated population of 2,704,810 in 2015. 
NTU OCW, like NTU Massive Open Online Courses (MOOCs), is one of the e-Learning and open 
educational resources (OER) initiatives at NTU, in responding to the global open education movement 
inspired by MIT OCW. Like other peer institutions, NTU tried to determine how to create open educational 
resources in general, and OCW in particular, to present the university’s unique features and core mission 
to global and local communities. Through research, discussions, and experiments across units on campus, 
the university has identified issues and challenges of an OCW project and translated them into action plans. 
It became apparent that a task force and a model were needed to overcome these issues and to carry out the 
plans. One immediate action was to form an OCW working group in 2011 to coordinate tasks between and 
among multiple units across the university. The same group is responsible for carrying out OCW plans, 
resulting in the current NTU OCW site/project. 

NTU-OCW Working Group 

Conversations between and among multiple units on campus revealed a need to have a unified unit or 
working group to efficiently and effectively utilize university resources as well as enhance/ensure course 
quality (although it is access-free). The primary role of this working group is to ensure success of the NTU 
OCW project. This involves multiple layers of planning, organization, and execution. Strategic planning and 
execution tasks include: 

• Course selection criteria and quality standards. 

• Operating (production) procedures. 

• Faculty buy-in and involvement. 

• Unit coordination and collaboration. 

• Advocacy of open education and promotion of OER/OCW awareness. 

The NTU OCW working group is a cross-unit project group. Members include Center for the Advancement 
of Science Education (CASE), College of Liberal Arts, NTU Press, Center for Teaching and Learning 
Development (CTLD), and Division of Computer Information Management (DCIM) in the Office of 
Academic Affairs. The working group is coordinated by the Dean of the NTU Office of Academic Affairs. 
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CASE, College of Liberal Arts, NTU press, and CTLD are responsible for course creation and production. 
DCIM, the Division of Computer Information Management in the Office of Academic Affairs, is in charge 
of development and maintenance of the project website (see Figure 1). The Division of Learning Technology 
(DLT), a sub-unit in CTLD, was primary working unit dedicate to OCW project. Besides in charge of 
approximately 80% of OCW course production, the unit coordinate tasks between course production and 
the website, provide instructional design consultation to faculty and student production assistants (PA) 
training. (See Figure 1) 

The project group meets monthly to track working progress and to discuss issues related to effectively 
integrating cross-unit manpower to allow the project to move forward quickly. In addition, standardized 
production processes for inviting and producing courses were established, in the early stages of the project, 
so group members could follow implementation. Furthermore, these standards were used as the basis for 
tracking and evaluation of the project to ensure the effectiveness of its implementation. 



Figure l. Organization structure and primary responsivity. 

Features of NTU OCW Implementation 

The features of NTU OCW are the result of strategies used for overcoming challenges or barriers while 
implementing OCW at NTU. 

1 . Integrating existing resources and establishing standard operating procedures. 
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Like most OCW projects, adequate financial and other resources are major challenges. To start the project, 
NTU OCW integrated all existing resources in the university, including administrative support, multimedia 
production resources (skills, manpower and equipment), and professional expertise on internet intellectual 
property. As a result, the NTU OCW working group, a unified cross-unit project group, was formed to 
coordinate efforts and carry out the mission. For the same reason, operating procedures were created to 
ensure quality and smooth workflow between units. 

2. Setting course selection criteria featuring the strength of NTU and Taiwan. 

With limited sources and an unwillingness to compromise quality, NTU-OCW needed to be selective in 
terms of the course selection. Rather than open to all courses, NTU-OCW developed course selection criteria 
that feature the strength and specialty of NTU, such as “Dream of Red Chamber” in classic Chinese literature, 
“I-Ching,” “Zhuangzi” in Chinese philosophy, and Sociology of Love. These courses were the most popular 
on campus. The quality of courses and scholarship of the faculty instructors have been recognized in 
appropriate subject areas. The criteria also consider what would benefit the largest audience, such as 
general education courses, core courses, and fundamentals in several professions. In addition, NTU OCW 
invited distinguished teaching faculty members who received NTU outstanding or excellent teaching 
awards to participate. 

3. Providing coherent program support to enhance faculty participation. 

Faculty members play essential roles in any course, including OCW courses. Unfortunately, low faculty 
participation is a common challenge for the creation and adoption of OER. Possible reasons include a lack 
of related experience or skills (i.e., digital media development), the quality of online courses, intellectual 
property concerns about course content, and extra workload to an already busy schedule. These are 
common barriers in most e-learning instruction (online materials, distance programs, and blended 
learning). Solutions include simplifying production process with templates or models that ensure quality 
and effectiveness of products, providing support in copyright licensing, offering incentive and grant 
opportunities for contribution, and moving as much production as possible to the classroom where 
instructors teach, rather than going to the studio. These steps are intended to reduce teachers’ common 
concerns and increase participation. 

4. Strictly following the Creative Commons license. 

One of the important features of OER and OCW is to freely open its content to all learners. To make the 
materials freely open, accessible, and used by others for educational purposes, Creative Commons license 
must be implemented. This allows educators and researchers to use the content for educational purposes, 
to modify it to create more diverse elements and usages, and to disseminate or share it. Under this premise, 
the intellectual property of OCW course materials should be protected by strictly following relevant laws 
and regulations. The most common type of CC license requested by faculty members is CC BY-NC-SA, 
Attribution-NonCommercial-ShareAlike, which lets others remix, tweak, and build upon their work only 
for non-commercial purpose. As a result, course materials can be adopted in more diverse ways and 
disseminated/shared to wider usage for educational purposes. All materials were reviewed by the 
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production team, including a copyright lawyer, with the “copyright procedure” before uploading to the OCW 
site (see Figure 2.) 



Figure 2. Copyright procedure. 

NTU OCW Production Model 

The Division of Learning Technology, a sub-unit in CTLD, and CASE are responsible for course production 
and for dealing with intellectual property materials. To meet goals and overcome challenges, the Center of 
Teaching and Learning Development established a working model for OCW creations and formed a team 
designated to carry out the OCW mission. While other operational layers are important, this paper focuses 
on the production layer, the essential part of the NTU-OCW working model and another key factor in NTU’s 
success. Two important and successful strategies are: (1) established operating procedures, and (2) a 
designated ID-team approach. 

Operating procedure: All NTU OCW courses contain comprehensive content materials including video clips, 
hangouts, and course materials. High-standard intellectual property regulation and Creative Commons (CC) 
are applied to all NTU OCW course materials. The creation of one OCW course is a continuing process 
throughout a semester (or a process of more than 22 weeks, including production assistant (PA) training, 
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faculty consultation, preparation, and wrapping up). All NTU-OCW courses include course contents of a 
full semester (18 weeks). There is a need for well-designed operating procedures to achieve efficient 
workflows and organization with multiple tasks/courses across units. Such procedures also ensure the 
quality of final production materials. Instructional design principles were applied. Instructional 
consultation to faculty were provided before the semester started to ensure best results in transferring the 
course to an online environment. 

Designated team approach: a designated trained production team was assigned to each faculty for a specific 
OCW course. A team usually consisted of two student PAs supervised by the CTLD staff with a background 
in instructional design and in assisting faculty in two major areas: media production and intellectual 
property/CC. For a 3-credit hour course, the average workload for a student PA is 5-10 hours per week. 
Each CTLD staff member typically supervises two or three courses per semester. Participating professors 
received support for his/her course. Combined with operating procedures, the designated and trained team 
approach represents the quality control gatekeeper. 

Faculty Support 

As mentioned earlier, faculty members have essential roles in the completion process. Support and 
resources were provided to ensure that the participating faculty was comfortable and free of concern. 
Common support includes instructional design consultation and intellectual property consultation. 
Instructional consultation was provided to explain the production process and to discuss the best way to 
transfer face-to-face courses into an OCW course delivered online. Faculty was encouraged to use materials 
best suited to their teaching, including copyright restricted materials, open licensed materials, and 
resources of their own creation. The production team provided proper support regarding copyright issues. 

Student Involvement 

To make the OCW project more financially sustainable, NTU OCW recruited part-time production 
assistants (PAs) to assist in the course production process. Production assistants (PA) are primarily 
students with job responsibilities in two categories: (1) Intellectual property assistance for assisting 
professors in creating course content and materials (such as presentation files, and handouts), and to 
double check content source and copyright notices; and (2) Media production assistance for assisting in 
the OCW media production process, including set-up, video shooting, and editing of course sessions. This 
is a key factor in the success of NTU-OCW and in making NTU OCW more financially sustainable. The 
working experience also provides students with opportunities to gain more understanding of course content. 
Students learn technical skills and deepen their knowledge as a result. 

Tips for part-time PA recruitment and training. NTU OCW usually suggests professors recommend 
their own students or students who took the course previously to assist in handling intellectual property. 
These students are familiar with course content which can enhance communication with faculty instructors 
and the efficiency in processing course content and confirming sources. The same concept applies to media 
PA recruitment. NTU OCW recruits students interested in OER/OCW and have multi-media experience to 
be media assistants. These students are more likely to complete their tasks successfully. 
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Before starting work, all part-time PAs must complete two stages of training. First, before the beginning of 
each semester, an all-day training session is required which includes an introduction to open education, the 
basic concepts of copyright and CC license, an explanation of the work involved, and practical drills. This 
training usually takes place one week before the semester starts. It allows PAs to complete the training and 
quickly apply what they have learned in their new tasks. The second stage of training occurs in the first few 
weeks of the course, when full-time staff/assistants sit in student classrooms to guide PAs toward being 
able to work independently. 


The Use of NTU OCW 

This section reports the results of the investigation on the use of NTU OCW. Data from Google Analytics 
was reviewed to gain a better understanding of the uses, characteristics, course preferences, and behaviors 
of NTU-OCW users. Specific areas for investigation include the growth of the site, type of users, users’ 
devices, length of viewing, and demographical distribution. Data was collected and analyzed to answer the 
following questions: 

• How many courses were created? 

• How many visitors accessed the sites? 

• What percentages of users/students were new versus returning students? 

• What kinds of devices were to access NTU-OCW? 

• What were the characters and viewing behaviors of users? 

• How can the findings guide NTU for next steps related to OCW and other forms of open education? 

Growing Number of Courses, Visitors, Faculty Involvement 

Since 2011, 213 high quality OCW course materials were completed and available for users on the NTU 
OCW website. On average, approximately 43 new courses were added to NTU OCW each year. In 2016, 
OCW site reached the 12 million visitors mark. More than 100 faculty members have participated in the 
project. Several devoted professors have more than two courses on the NTU OCW. 

Before the NTU OCW working group was established in 2011, NTU had 31 online courses created by 
different centers and departments, including CASE, GET, and NTU Open Forum for New Intellectuals. New 
productions were started by the OCW working group in late 2011. In 2013, special attention was given to 
creating higher quality materials in video, audio, and comprehensiveness of course content. (Figure 3) 
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Figure 3. The number of courses added in NTU OCW each year. 

All NTU OCW courses are in Chinese, and almost half are in Liberal Arts (46%). As one of the leading 
universities in a Chinese-speaking area, NTU shares the responsibility to preserve Chinese culture and 
provide quality open educational resources to its world-wide audience. Courses are selected based on 
criteria that highlights the strengths and characteristics of NTU. Other than Liberal Arts, 24% of courses 
are in General & Applied Science, 19% courses are in Social Science, and 11% courses are in Natural & 
Medical Science (Table 1). 

Table 1 

Disciplines of NTU OCW Courses 


Course name 

% 

Liberal Arts 

46 

Social Science 

19 

General & Applied Science 

24 

Natural & Medical Science 

11 


The five most popular NTU courses are The Chamber of the Red Dreams from the Department of Chinese, 
Calculus from the Department of Mathematics, Introduction to Phonetics from the Department of Foreign 
Languages, Sociology of Love from the Department of Sociology, and History of Chinese Literature from 
the Department of Chinese. It is important to point out that professor Li-Chuan Ou of the The Chamber of 
the Red Dreams, has thousands of fans in the Chinese region. Her continuous devotion to open education 
has resulted in the highest viewed OCW courses. She has an extensive fan base and has won the Global 
Open Education Alliance award of the Open Education Awards for Excellence Teaching Award (Educator 
Award) in 2015. 
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Characteristics of OCW Audience (Site Visitors) 

In 2016, the NTU OCW website exceeded 12 million visits. On average, there are 9,700 sessions every day 
(see Figure 4). Visits have increased from 800 to 8,000 daily, a clear demonstration that NTU OCW 
website's visibility and popularity are growing. 

4,000,000 

3,500,000 
3,000,000 
2,500,000 
2,000,000 
1,500,000 
1,000,000 
500,000 
0 

Figure 4. Sessions of NTU OCW users on the website each year. 

Table 2 shows the characters of the NTU OCW users in terms of age group and gender. About one third of 
them are in the 18-24 age group (34%), 29% in the 25-34 age group, and 16% in the 35-44 age group. Elderly 
users 65 and older constitute 3.58% of all users. Slightly more users are female (51%), while 49% of users 
are male. 

Table 2 


Age and Gender of NTU OCW Users 


Characteristics 

% 

Age group 

65+ 

4 

55-64 

8 

45-54 

9 

35-44 

16 

25-34 

29 

18-24 

34 

Gender 

Male 

49 

Female 

51 


3,601,242 



2012 2013 2014 2015 2016 
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More than half (67%) of NTU OCW users are returning visitors, indicating a stable part of the audience 
continues to view online courses (Figure 5). It is encouraging to learn visitors are returning, which could 
indicate continued learning, one of primary missions of the OCW. This is especially true for the “non-school” 
age group (typically older than 25 years old unless they are pursing advanced degrees). 



■ Returning Visitor ■ New Visitor 


Figure 5. Type of NTU OCW visitors. 

The geographical distribution of NTU OCW is mostly in Chinese speaking countries, Taiwan, China, and 
Singapore. As shown in Figure 6, 90% of all users are from Taiwan and China in most years. Because all 
NTU OCW courses are in Chinese and only few have English captions, it is understandable that the majority 
of users are from Chinese speaking countries. The difference in percentages between Taiwan and China 
users also indicates another interesting fact. Most of users of NTU MOOCs (massive open online courses) 
are from China, while most users of NTU-OCW are from Taiwan. It seems to indicate that two different 
types of online courses attract different groups of users. In addition, in recent years, the annual percentage 
of Chinese users declined from 28% to 14%. Possible reasons may be the overall enhancement of online 
learning courses in China in terms of course quality and quantity. For Taiwanese viewers, the numbers were 
steady at an average of three quarters of users while the total number of users and courses increase each 
year. Most are from metropolitan areas, which might indicate a digital divide between urban and rural 
areas. 
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2011 2012 2013 2014 2015 2016 

■ Taiwan I China lUSA Bothers 


Figure 6. Demographical distribution of NTU OCW users. 

According to GA, the average pages/session for each user is 3.87. This means most users of NTU OCW go 
almost directly to the course page they want or they leave the site after viewing four pages. Users in the 35- 
44 age group usually view the most pages (3.50) in each session, and users in the 18-24 age group view the 
least pages (2.95) in each session (Table 3). There is no difference in average page viewing between male 
and female users. In addition, the average time/session for each user is 4:35 minutes. As Table 3, shows 
users in the 18-24 age group spent the shortest time in each session, while elderly users in the 65+ age group 
spent the longest time in each session. Although there is a difference between age groups, no significant 
difference is suggested. 

Table 3 

Average Pages and Time Different Users View/ Spend on Each Session 


Age group 

Avg. pages 

Time (minutes) 

65+ 

3-41 

05:26 

55-64 

3-13 

04:15 

45-54 

3-15 

04:11 

35-44 

3-5 

04:56 

25-34 

3-29 

04:17 

18-24 

2-95 

03:51 


According to GA, most viewers access NTU OCW by desktop computers, followed by mobile devices (such 
as cell phones). Overall, 72% of users use a desktop, 19% of users use mobile devices, and 9% use a tablet. 
However, as Figure 7 shows, the percentage of desktop use has decreased significantly and mobile device 
use has increased significantly in the past five years. When NTU OCW began in 2011, 94% of users used 
desktops to access the website. Only 3% of users used mobile devices. This trend toward the use mobile 
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devices to access online learning materials suggests that designing for multiple devices must be considered 
in the future. 



2011 2012 2013 2014 2015 2016 

■ desktop I mobile ■ tablet 


Figure 7. Devices NTU OCW users use to access the website. 


Conclusion 

Overall, the NTU OCW was successful despite the many challenges detailed in this study. The emphasis on 
quality was the primary factor in its successfully attracting visitors and supporting learning. NTU OCW has 
won the best OCW site by the Global Open Education Alliance in 2014. An OCW professor won the Open 
Education Award for Excellence Teaching Award in 2015. Most importantly, in 2016, the NTU OCW website 
successfully attracted more than 12 million individual visits from primarily lifelong learners (65%) in 
informal learning settings. The number of visitors has increased consistently each year, with the ratio of 
returning users steady since launching in 2010. 

Statistics indicate an overall successful use of NTU-OCW for Chinese speaking users, especially in urban 
areas where information and communication technology is more developed. We can conclude that a digital 
divide exists, and it is one of challenges that OER faces. We also can conclude that materials are limited to 
Chinese speaking users because all courses are in Chinese and only a few have English translations. 
Although it is not necessary to have all materials translated, doing so with elective courses could interest 
learners world-wide. Because Chinese literature is one of most popular subject areas and a unique 
contribution to the world audience, some works have been translated, with subtitles in English. This could 
be beneficial to the world learners and scholars who want to learn more Chinese literature. 

Quality Matters 
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All OCW courses are complete with 18 weeks of lecture and all corresponding course materials. The 
dedication to high-quality course production and consistent efforts in processing materials according to 
high standards of intellectual property regulation also helped NTU gain a solid reputation in the global 
community of open education. In 2014, the NTU OCW website was awarded the “Outstanding site” award 
by the Open Education Consortium. The award recognition by the open education consortium as the best 
OCW Website indicates the design and quality control mechanisms were a successful model as well. NTU- 
OCW will continue using this model for future OCW courses. 

Team Efforts 

Team effort is another key factor in success. Regardless of the challenges, issues, and limitations, NTU OCW 
has achieved its initial goal to successfully provide whole courses with high quality open educational 
materials. It requires collaboration and team efforts across units, including administrative support from 
the university, the Dean of the Office of Academic Affairs, the Director of NTU Open Forum for New 
Intellectuals and the Director of CASE, and CLT. Participating professors’ openness and enthusiasm for 
education to all is also an important factor in project success. More than too professors now participate in 
NTU OCW and share a devotion to open courseware. NTU OCW will continue to meet new challenges going 
forward. 

Future Directions 

With support from the university and many participating teachers, NTU OCW achieved most of its initial 
goals. However, with the recent emergence of MOOCs and various but significant funding reductions, NTU 
OCW has an urgent need to start thinking of new goals and strategies as its next steps. 

Because so much effort, time, and resources have been expended to create a significant number of quality 
OCW courses, their wide spread use is important. A natural next step will be to promote OCW as valuable 
OER that can be accessed online freely for educational purposes. For OCW promotion, an array of 
information will be provided steadily at activities inside and outside of campus. To actively promote OCW 
courses to students in school, i-minute promotional clips for selected courses are to be produced and 
broadcast on digital signage in school buildings and on the NTU channel of YouTube EDU. 

These promotional clips usually include short footage devoted to course lectures or images of course 
materials. Some interesting questions related to the course content are included in the clips, encouraging 
students to visit the website, find the complete course videos, and obtain answers. Furthermore, in order to 
reach a wider audience outside the school, pieces of course sessions are also broadcast on the NTU channel 
of YouTube EDU. Members of the audience who watch the course sessions and want to learn more can be 
linked to the complete course page on the NTU OCW website. Social media such as Facebook is also used 
to promote course content and the OCW website. Although many positive responses have been received, 
the possibility of attracting a new and different demographic audience remains questionable. 

For future e-learning, NTU OCW wants to effectively increase the usage of course content. Possible practices 
may include research and development of distinct ways to distinguish content management for different 
online courses. For example: OCW and MOOCs users have different learning needs and patterns, which 
might require that different instructional designs be applied. The re-use value of OCW materials should be 
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further developed, such as sorting and indexing the teaching materials of OCW. It can make users search 
and access the needed information easier. 

As the number of courses grows, an index system needs to be established to assist users to search and find 
courses more efficiently. Other features, such as rating and review, can also assist users in choosing courses. 
NTU can provide information or workshops promoting re-use or re-create learning materials on the basis 
of OCW content and materials. New ways of innovative teaching and learning also can improve OCW values, 
such as finding ways to assist teachers in using OCW materials for flip-teaching and blended learning. 
Significantly, adding closed captions to courses for accessibility to all is on the list. Currently, only partial 
courses with closed captions provided by YouTube are available. This ensures that courses are accessible by 
users with special needs and helps reach a much wider audience. These myriad ways can increase the usage 
of open education resources for the benefit of learners, instructors, learning center personnel, and other 
interested parties. We are just beginning to experiment with and explore some of them. 
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